observations = X.iloc[[0, 2, 16, 1337], :]
observations
model.predict(observations)
exp = explainer.explain_instance(observations.iloc[0, :], model.predict, num_features=5)
exp.show_in_notebook(show_table=True)
exp = explainer.explain_instance(observations.iloc[1, :], model.predict, num_features=5)
exp.show_in_notebook(show_table=True)
exp = explainer.explain_instance(observations.iloc[2, :], model.predict, num_features=5)
exp.show_in_notebook(show_table=True)
exp = explainer.explain_instance(observations.iloc[3, :], model.predict, num_features=5)
exp.show_in_notebook(show_table=True)
display(observations.iloc[[0], :])
shap.plots.waterfall(shap_values[0], max_display=7)
display(observations.iloc[[1], :])
shap.plots.waterfall(shap_values[1], max_display=7)
display(observations.iloc[[2], :])
shap.plots.waterfall(shap_values[2], max_display=7)
display(observations.iloc[[3], :])
shap.plots.waterfall(shap_values[3], max_display=7)
The main differences seem to be that shap values give insight how particular feature changes the average value for a given observation. Lime on the other hand because of fitting a linear model under the hood only can indicate if a feature correlates positively and what is the magnitude of the change. Looking at the last example - for low value (70) of reactions SHAP lowers the estimation, but LIME just says that higher value would result in higher prediction (the average is hidden in the intercept).
exp = explainer.explain_instance(observations.iloc[0, :], linear_model.predict, num_features=5)
exp.show_in_notebook(show_table=True)
exp = explainer.explain_instance(observations.iloc[1, :], linear_model.predict, num_features=5)
exp.show_in_notebook(show_table=True)
exp = explainer.explain_instance(observations.iloc[2, :], linear_model.predict, num_features=5)
exp.show_in_notebook(show_table=True)
exp = explainer.explain_instance(observations.iloc[3, :], linear_model.predict, num_features=5)
exp.show_in_notebook(show_table=True)